gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded by maurycy · Pull Request #153365 · python/cpython

maurycy · 2026-07-08T20:09:48Z

The PR hardens and cleans up frame, coro and task-waiter walks a bit, by making them iterative and bounded.

I'm addressing two issues here:

No walk limits. The walks use pointers read from the target without any guarantees, so a torn read could stitch a cycle. Not to mention adversial targets. process_frame_chain() already had 1024 + 512 limit, but the PR extracts it as MAX_FRAME_CHAIN_DEPTH and applies in the parse_coro_chain(), used by both sync and async paths, and parse_async_frame_chain(). The task-waiter walk in get_async_stack_trace() gets a separate limit (MAX_TASK_WAITER_WALK_TASKS), on the total number of tasks visited (including duplicates), since its' a graph, not a chain.
Stack overflows. Both coro and task-waiter walks were recursive. With 4KB (SIZEOF_TASK_OBJ) 256 was enough to overflow a 1MiB stack.

The obvious context here is avoiding infinite loops. Truth be told, I think that in some places torn reads were also responsible for avoiding them and exists. :-)

Issue: _remote_debugging: No frame limit in get_async_stack_trace() #153364

maurycy · 2026-07-08T21:50:56Z

 #define MAX_STACK_CHUNK_SIZE (16 * 1024 * 1024)  /* 16 MB max for stack chunks */
 #define MAX_LONG_DIGITS 64  /* Allows values up to ~2^1920 */
 #define MAX_SET_TABLE_SIZE (1 << 20)  /* 1 million entries max for set iteration */
+#define MAX_FRAME_CHAIN_DEPTH (1024 + 512)  /* Iteration bound for frame chain walks */


Not the scope but maybe MAX_THREADS, MAX_STACK_CHUNKS, MAX_LINETABLE_ENTRIES or MAX_ITERATIONS are worth keeping here too

pablogsal · 2026-07-11T14:25:36Z

+    PyObject *result,
+    size_t depth
 ) {
+    if (depth >= MAX_TASK_WAITER_CHAIN_DEPTH) {


This can still overflow a 1 MiB stack: every recursive level keeps the 4 KiB task_obj alive, so 256 levels exhaust the stack before this check fires. We should walk the waiter graph iteratively.

@pablogsal I've changed both to iterative.

bedevere-app · 2026-07-11T14:25:42Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

maurycy · 2026-07-13T07:50:14Z

I have made the requested changes; please review again.

bedevere-app · 2026-07-13T07:50:19Z

Thanks for making the requested changes!

@pablogsal: please review the changes made to this pull request.

maurycy · 2026-07-13T08:00:32Z

                return -1;
            }
-
-            if (ref_cnt) {


I believe this was a bug:

this address is already read as key_addr: https://github.com/python/cpython/pull/153365/changes#diff-d3a9f5049ca6c549a4feb928b6d06a7dbce8486745bb06f6b8254cddce207e32L150 (which is correctly named)

if we aim to filter out tombstones, then we should use dummy or hash != -1:

cpython/Objects/setobject.c

Line 595 in c843cb7

FT_ATOMIC_STORE_PTR_RELEASE(entry->key, dummy);

cpython/Include/cpython/setobject.h

Lines 5 to 22 in c843cb7

/* There are three kinds of entries in the table:

1. Unused: key == NULL and hash == 0

2. Dummy: key == dummy and hash == -1

3. Active: key != NULL and key != dummy and hash != -1

The hash field of Unused slots is always zero.

The hash field of Dummy slots are set to -1

meaning that dummy entries can be detected by

either entry->key==dummy or by entry->hash==-1.

*/

#define PySet_MINSIZE 8

typedef struct {

PyObject *key;

Py_hash_t hash; /* Cached hash code of the key */

ref 1885988

maurycy · 2026-07-13T08:43:17Z

+        PyObject *task_info = PyList_GET_ITEM(result, i);
+        PyObject *waiters = PyStructSequence_GET_ITEM(task_info, 3);
+        for (Py_ssize_t j = 0; j < PyList_GET_SIZE(waiters); j++) {
+            if (PyList_GET_SIZE(result) >= MAX_TASK_WAITER_WALK_TASKS) {


Not keeping the visited set, so duplicated are counted towards. I believe that 65536 gives enough head room?

maurycy · 2026-07-13T08:49:51Z

-int
-process_task_and_waiters(
+static int
+process_task_waiters(


BFS. Leveraging results as the queue. Note that the current order is not documented and not relied by tests.

maurycy · 2026-07-14T12:53:42Z

-    void *context
-);
-
 extern int process_single_task_node(


It's used only in the asyncio.c. Does it, and friends, need to be in the header?

maurycy added 4 commits July 8, 2026 21:36

let me declare single limit

4dd6f49

use our new limit in process_frame_chain()

e71aaa5

add it in parse_async_frame_chain()

116da45

parse_coro_chain()

3e12939

maurycy requested a review from pablogsal as a code owner July 8, 2026 20:09

bedevere-app Bot added the awaiting review label Jul 8, 2026

bedevere-app Bot mentioned this pull request Jul 8, 2026

_remote_debugging: No frame limit in get_async_stack_trace() #153364

Open

NEWS

c46810c

maurycy commented Jul 8, 2026

View reviewed changes

Comment thread Modules/_remote_debugging/asyncio.c

maurycy added 3 commits July 8, 2026 23:07

async in the message?

dd6e19a

test

0972e00

no race

3c5f78b

maurycy commented Jul 8, 2026

View reviewed changes

maurycy added 4 commits July 10, 2026 21:25

process_task_awaited_by

78c18f9

process_task_awaited_by limit test

21d764e

NEWS

000287e

MAX_TASK_WAITER_CHAIN_DEPTH

353f815

maurycy changed the title ~~gh-153364: Add frame limit in get_async_stack_trace() and parse_coro_chain()~~ gh-153364: Limit frame, coroutine, and task-waiter chain walks Jul 10, 2026

maurycy added 2 commits July 10, 2026 22:39

TASK_WAITER_CHAIN_DEPTH in test

f66866b

TASK_WAITER_CHAIN_DEPTH 256

c9f1ebd

pablogsal requested changes Jul 11, 2026

View reviewed changes

bedevere-app Bot removed the awaiting review label Jul 11, 2026

bedevere-app Bot added the awaiting changes label Jul 11, 2026

maurycy added 6 commits July 11, 2026 20:54

prevent the drift with the comment

7236ca2

better naming, better style

35fc6c6

MAX_TASK_WAITER_CHAIN_DEPTH comment

10149e9

task-waiter iterative bfs walk

ef51563

iterative coro-walk

b5022ba

nicer news

73b8674

bedevere-app Bot added awaiting change review and removed awaiting changes labels Jul 13, 2026

bedevere-app Bot requested a review from pablogsal July 13, 2026 07:50

maurycy changed the title ~~gh-153364: Limit frame, coroutine, and task-waiter chain walks~~ gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded Jul 13, 2026

maurycy commented Jul 13, 2026

View reviewed changes

maurycy commented Jul 14, 2026

View reviewed changes

	/* There are three kinds of entries in the table:

	1. Unused: key == NULL and hash == 0
	2. Dummy: key == dummy and hash == -1
	3. Active: key != NULL and key != dummy and hash != -1

	The hash field of Unused slots is always zero.

	The hash field of Dummy slots are set to -1
	meaning that dummy entries can be detected by
	either entry->key==dummy or by entry->hash==-1.
	*/

	#define PySet_MINSIZE 8

	typedef struct {
	PyObject *key;
	Py_hash_t hash; /* Cached hash code of the key */

Uh oh!

Uh oh!

Conversation

maurycy commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

maurycy Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pablogsal Jul 11, 2026

Choose a reason for hiding this comment

Uh oh!

maurycy Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented Jul 11, 2026

Uh oh!

maurycy commented Jul 13, 2026

Uh oh!

bedevere-app Bot commented Jul 13, 2026

Uh oh!

maurycy Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maurycy Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

maurycy Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maurycy Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maurycy commented Jul 8, 2026 •

edited

Loading

maurycy Jul 8, 2026 •

edited

Loading

maurycy Jul 13, 2026 •

edited

Loading

maurycy Jul 13, 2026 •

edited

Loading

maurycy Jul 14, 2026 •

edited

Loading